This invention is concerned with the field of three-dimensional imaging and in particular with the provision of a natural three-dimensional image of an operative site via, inter alia, an endoscope, thereby improving the eye-hand co-ordination of a surgeon, whilst he/she carries out such a surgical procedure.
The field of Minimally Invasive Endoscopic Surgery (MIES) has, for a period in excess of 10 years, been dependent on the use of small diameter viewing devices, collectively known as endoscopes. Some of these are flexible and a majority rigid in construction. A typical working length of the insertion portion of such an instrument would be in excess of 30 cm. The diameter of this part of the instrument will typically vary from 10 mm to as low as 1 mm or less. Small diameter optics or a coherent fibre-optic bundle transfers an image of the operative site, via the distal tip of the insertion portion located inside the patient, to an external CCD camera. The image acquired in this way is displayed on a conventional television monitor placed at a comfortable viewing distance from the surgeon.
We are used to viewing live imagery on a television screen from a distance of between 6 to 12 feet. Such a screen is of considerable size and typically provides a convenient viewing window, at such a distance. Conventional television quality is sufficient to provide the definition required within this window for such passive viewing.
It has been accepted for some considerable time that a conventional two-dimensional (flat) image is a poor substitute for the natural binocular vision enjoyed by a surgeon, when carrying out an open surgical procedure. Nevertheless, the benefits to the patient, in terms of reduced trauma and scarring, and the reduced cost to the hospital, in terms of the patient""s stay, have fuelled considerable growth in the MIES market, with one type of relatively simple procedure, the laparoscopic cholesystectomy, being carried out routinely, today, in this manner. Unfortunately, the handicap imposed on the surgeon""s dexterity (eye-hand co-ordination) by the impaired vision provided through a conventional video image has been a brake on the migration of this technique to a variety of more complex procedures. Accordingly, the need to provide a better solution and, in particular, a three-dimensional image of the operative site has been an accepted goal, for a number of years. However, a variety of commercial attempts to address the problem, have met with poor acceptance by the surgical profession, and it has been the recognition of this reality that has provided the incentive to analyse why such first generation three-dimensional visualisation systems failed. As a result of this analysis, a second generation three-dimensional endoscopic visualisation system has been developed and this is the subject of the current invention.
In order that the scale of the problem and, thus, the significance of the improvement provided by the current invention may be fully appreciated, a summary of the analysis of the shortcomings of a typical first generation three-dimensional endoscopic system is provided here.
A typical configuration of a first generation 3D endoscopic visualisation system might include.
A TV monitor with a picture diagonal of between 14xe2x80x3 and 20xe2x80x3
b) An endoscope which provides two points of view, effectively spaced a fraction of a millimetre apart at the distal tip. This may be described as a stereo-endoscope, although conventional systems have been used to achieve this to a small degree.
c) Special glasses which, when worn by the user, allow sequential viewing, respectively by left and right eye, of the two alternative perspective views of the scene, displayed sequentially on the monitor.
In use, the preferred working distance of the distal tip would be between 30 and 90 mm from the subject matter of interest, and the preferred viewing distance of the monitor would be at a conventional 2 to 3 metres from the observer. A substantial 3D effect would be observed, but, after some time, the observer would begin to feel uncomfortable and, in some cases, would suffer severe nausea. Eyestrain and headaches would be commonplace. In addition, the improvement in eye-hand co-ordination was not always as great as was expected.
In order to analyse the reasons for the effects described, it is useful to compare the nature of the binocular imagery provided to the observer by such a first generation system with that which would be experienced in a direct vision situation, and, in particular, when such a situation involves the manipulation of surgical instruments.
Historically there have been a number of factors which have created problems. These included:
1) The image acquisition geometry of the stereo-endoscope failed to adequately match the observer""s viewing geometry;
2) The left and right eye images were displayed field sequentially, and not simultaneously as would be the case for natural vision;
3) The stereo-monitor displayed an image intended to aid eye-hand co-ordination, but this was not presented at a viewing distance, typically arm""s length, where we would normally expect to find an object which we are manipulating;
4) The brightness of the image displayed was compromised, as a result of the field sequential display process.
It is a tenet of the current invention that the optimal solution for providing a more natural three-dimensional image, suitable for effective eye-hand co-ordination, will include the avoidance or substantial reduction of all of the above issues.
On an individual basis, some of the above problems are addressed within the prior art. For instance, in U.S. Pat. No. 5,712,732 (Street) apparatus is described in which two two-dimensional perspective images, provided by conventional liquid crystal display (LCD) panels, are combined with the aid of a semi-transparent mirror, so that each eye of the observer sees a different perspective, but in the same location. This causes the brain to fuse these perspectives into one three-dimensional image. Observer tracking ensures that each eye only sees the image intended for it. The principal purpose of the aforementioned invention is to avoid the need for the observer to wear special spectacles. Furthermore, it is normal for direct view LCD""s to be provided with a polarising layer on the side viewed by the observer. The direction of polarisation of this sheet of material is normally set at 45xc2x0 to the vertical. The use of a polarising element in front of each of two displays positioned mirror-symmetrically, either side of a semi-transparent mirror, is well known from the prior art. One such reference may be found on pages 365-366 of xe2x80x9cThree-Dimensional Imaging Techniquesxe2x80x9d by Okoshi and published in 1976 by Academic Press Inc. By using suitably polarised spectacles, the viewer may therefore observe a stereoscopic or three-dimensional image. In both of these examples the observer is provided with left and right eye images, simultaneously. In International Patent Application PCT/GB97/00766 (Street) the problem of barrel distortion, encountered when using wide angle optics, is identified as causing undesirable (for instance vertical) disparity errors in a twin axis stereo-endoscope. The inventive step of PCT/GB97/00766 is to compensate for this problem by matching the distortions of both optical channels so that this effect is minimised.
It is an object of the current invention to provide a vision system which enables an observer to relate to an object field through a three-dimensional image scaled in proportion to said object field, so as to provide an environment for comfortable eye-hand co-ordination. More specifically, the invention is intended to provide a generalised method for improving eye-hand co-ordination, during videoscopic surgery.
It is a further object of the invention to provide the three-dimensional image substantially free from binocular disparity between corresponding points which would not be observed were the object field to be viewed directly.
It is also an object of the invention to provide the observer with both left and right eye images simultaneously, in order to more effectively simulate the conditions of normal binocular viewing of an object field.
It is another object of the invention to minimise the conflict between accommodation and convergence by providing suitable viewing conditions for the observer.
Thus according to the invention, a system for relaying a three-dimensional view of an object field to an observer comprises:
means for acquiring a first image of the object field from a first viewpoint and a second image of the object field from a second viewpoint, in which the viewpoints are spaced, so that, in use, a line joining the viewpoints subtends a first angle at an object point within the object field, the first angle lying within a range of three degrees to sixteen degrees;
means for displaying an image of the object point within an image field, said display means being arranged to present simultaneously at a plane a first relayed image of the object field acquired from the first viewpoint for viewing by the left eye of the observer and co-located at the plane a second relayed image of the object field acquired from the second viewpoint for viewing by the right eye of the observer; and
means for positioning the plane at a distance from the observer so that, in use, a line joining the two eyes of the observer subtends a second angle at a point on the plane which lies in the range of three degrees to sixteen degrees.
Preferably said display means includes a liquid crystal display.
Advantageously, the ratio of the first angle and the second angle lies within the range 0.5 to 2.
The positioning means may include an articulated arm.
The display means can include a manual adjustment such that the brightness of the image displayed may be adjusted to be substantially equal to or exceed that of the surrounding environment.
Optionally, the system may comprise:
a sensor for sensing ambient light in the vicinity of said display means; and
control means responsive to said sensor for controlling the brightness of the image displayed by said display means.
Advantageously, in use, peak white within the image viewed has a luminance in excess of 200 candela per square metre, whereby dilation of the pupil of each of the observer""s eyes is limited, the observer retains a good depth of field and conflict between accommodation and convergence for a stereoscopic image is mitigated and/or avoided.
Preferably, the peak white has a luminance in the range of 250 to 700 candela per square metre.
According to another aspect of the invention, in use and at the plane, for any point within the first relayed image which has a corresponding point within the second relayed image, the vertical component of the line joining said point within the first relayed image to said corresponding point within the second relayed image subtends a third angle at the observer which does not exceed 0.5 degrees. Preferably, the third angle does not exceed 0.1 degrees.
According to the invention, a method for relaying a three-dimensional view of an object field to an observer comprises:
spacing a first and a second viewpoint so that a line joining the viewpoints subtends a first angle at an object point within the object field, the first angle lying within a range of three degrees to sixteen degrees;
acquiring a first image of the object field from the first viewpoint and a second image of the object field from the second viewpoint;
simultaneously displaying at a plane a first relayed image of the object field acquired from the first viewpoint for viewing by the left eye of the observer and, co-located at the plane, a second relayed image of the object field acquired from the second viewpoint for viewing by the right eye of the observer; and
positioning the plane at a distance from the observer so that, in use, a line joining the two eyes of the observer subtends a second angle at a point on the plane which lies in the range of three degrees to sixteen degrees.
Advantageously, the ratio of the first angle and the second angle lies within the range 0.5 to 2.
Preferably, the method further comprises:
adjusting the brightness of the image displayed to be substantially equal to or greater than that of the environment.
Advantageously, the brightness is adjusted to exceed 200 candela per square metre. It may be adjusted or controlled to fall within the range of 250 to 700 candela per square metre.
Further scope of applicability of the present invention will become apparent from the detailed description given herein after. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description.